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Abstract 

We prove an ergodic theorem for a linearwise Markov process, and we give estimation for 
the average of renewal process amount of overjump; these results are based on Smith’s Key 
Renewal Theorem. 

Also we discuss the possibility of using these results for studying some type of queueing 
systems. 


1 Introduction 

Smith’s Key Renewal Theorem is the basis for the development of Renewal Theory. This theorem 
implies many useful consequences, see, e.g., i. n. 0. etc. Here we present some consequences 
of Smith’s theorems useful for queueing theory. We prove the ergodic theorem for linearwise 
Markov process, as well as we find an estimation of the expectation of renewal process amount of 
underjump (or backward renewal time), and we give an estimation for the expectation of renewal 
process amount of overjump (or forward renewal time). 

This paper is organized as follows. 

In Section 2 we recall the basic facts of renewal theory, and formulate Smith’s Key Renewal 
Theorem in its most general form. 

In Section 3 we recall the notion of linearwise (or lineartype) Markov process, formulate and 
prove the ergodic theorem for linearwise Markov process. 

In Section 4 we give estimation for renewal process amount of overjump expectation. 

In Section 5 we discuss the possibility of using the results of Sections 3-4 to prove the ergodic 
theorem concerning multidimensional piecewise-linear Markov processes. 

2 Smith’s Key Renewal Theorem 

2.1 Definitions 

Definition 1. Renewal process is a random increasing sequence 0 < < ^2 < • • • < A < • • •, 

where tj+i — ti = {i & N) are non-negative random variables with distribution function F{s), 
and ti is non-negative random variable G with distribution function T’i(s); random variables 
are mutually independent. The times ti are called renewal times (points) or jump times and the 
intervals (fj,tj+i) are called renewal intervals. 
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Remark 1. Sometimes renewal process is defined differently, bnt the meaning of these dehnitions 
remains the same - see, e.g., 9.9. 


Definition 2. Let N[t) = maxji : < iV} be a nnmber of renewals till a time t (or nnmber of 

d f ^ ^ \ 

jnmps observed np to some time t). Let ns denote H{t) = 'EM(t) = Fi{t) + Fi * ( ) i^)'i 

EN{t) = H{t) is called renewal function (here is a symbol of convolntion). 


Definition 3. A random variable is called lattice if all its possible valnes are in the set {a + nb, n G 
Z} = a + 6Z, fe 7 ^ 0. 

Definition 4. A non-negative fnnction g{x) : [0, -|-cxo) [0, -|-cxo) is called directly Riemann 

integrable if 


hmA> snp gf(x) = hmA> inf gf(x) G (-|-oo,-|-oo). 

^ xG[A(n-l),An] ^ xe[A(n-l),An] 

Definition 5. Let the seqnence 0 = to < < ^2 < • • • < < tj+i < • • • be a renewal process. 

For t > 0 denote by xl the time elapsing from the instant t to the next renewal, and by Xt the 
time elapsed from the previous renewal to t. Thus, if n is dehned by the condition ^ t < t„+i, 
then = tn+i — t, x{t) = t — t^- The variable x^ is called the amount of overjump (over the level 
t) and x{t) is the amount of underjump (up to level t) - see [2|, Ch. 2.7]. 

In ([H §2.1] or [m §2.1.1]) the amount of overjump is called “forward renewal time” or “residual 
life of the renewal interval”, and the amount of underjump is called “backward renewal time” or 
“elapsed time of the renewal interval”. 

In the sequel denote 'ip{x) = 1 — fj^x) for all functions fj^x). 


2.2 Smith’s Key Renewal Theorem 

Theorem 1 (Key Renewal Theorem). Let ( be non-lattice random variable with distribution func¬ 
tion <F(s) = P{C ^ s}, EC < 00 , H{t) = Yl ‘h”**(t); let b{s) be a directly Riemann integrable 

m=l 

function on [0, cxd). Then 

00 

t J b{s) ds 

lim / b(t — s) dH(s) = ° 


t—)-oo 


EC 


see m Theorem 35]. 


Remark 2. If F{s) is a distribution function, and / sdF{s) < 00 , then (1 — F{s)) is directly 

0 

Riemann integrable on [0, -|-cxo). 

Smith’s Key Renewal Theorem is very important for the renewal theory: for example, the 
following very useful fact (jU §5.2], [H §2.7.6] ) is a consequence of the Smith’s theorem; see also 
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Proposition 1. Consider the renewal process < ti < ■ ■ ■ < ti < ■ ■ ■, and let xt he the 

amount of underjump, and xl he the amount of overjump. 

OO 

If F{s) is non-lattice and f sdF(s) < oo, then 

0 

CO 

fF{,)ds 

^ -. ( 1 ) 

fF{,)ds 

0 

Corollary 1. Hence, 


lim P{xi > a} = lim Pjxj > a} 

t—)-00 t^OO 


lim E Xt = lim E x^ 

t^OO t—^OO 


OO 

°° f F(s) ds 


a 

oo 


0 /^(s)ds 
0 


da 


EC^ 

2EC’ 


Since processes describing the behaviour of queueing systems are often a combination of renewal 
processes in some sense, and asymptotic behaviour of these processes is very important, then 
renewal theory and, in particular, some facts similar to Proposition [T] and Corollary [T] are very 
signihcant. 

For example, for studying single server queueing system behaviour, we can use lineartype or 
linearwise Markov processes (0) with state space .21 = {Z_|_ x M+}. Linearwise process has 
two components. The hrst component is equal to a quantity of customers in queueing system, 
and the second component is equal to the time elapsed from last change of the hrst component: 

= {'nt,Xt)', let Ffc(s) be a distribution function of time intervals when the process {Xt, t ^ 0) is 
in the set {n* = k], k E 'Z+. 

Therefore, the following conjecture seems natural: if the distribution of a linearwise Markov 

OO 

process {Xt, t ^ 0) (weakly) converges to stationary distribution as t ^ oo, and if f sdFk{s) < oo 

0 

for all k G then stationary distribution of the process {Xt, t ^ 0) can be described by formulas 
similar to the formula m- 

However, usually description of multichannel (parallel servers) queueing system behaviour in 
terms of linearwise process is impossible: in this case state space of the process describing queue¬ 
ing system behaviour has a more complex structure. Nevertheless, B.A. Sevastyanov (see 0 . 0 ) 
proved an ergodic theorem for multichannel queueing system M|G|n|0 (Kendall’s notation - queue¬ 
ing system consists of n servers, arrival how is Poisson with parameter A, independent service times 
have an arbitrary distribution function G(s), and buher size equals to zero). B.A. Sevastyanov 
dehned the form of stationary distribution; this distribution is described by formulas similar to 
([I]); see also [S]. 

In some situations when service process has a stationary distribution, it is impossible to hnd 
any precise formula for this distribution: it is possible to hnd only an estimate for this stationary 
distribution. However, corresponding estimate can be useful not only for solving practical problems 
related to optimization of queueing system work. Estimation of stationary distribution also can 
be used to estimate convergence rate to stationary distribution for the distribution of process 
describing the behaviour of queueing system. 

Below we give some facts, which can be used to obtain estimates of stationary distribution. Also 
these facts can be used for estimating the convergence rate of some queueing process distribution 
to this stationary distribution. 
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For example, in [Hj a strict (not only qualitative) estimate of convergence rate for availability 
factor has been obtained with the help of Theorem [2] and Lemma [H 

These facts (Theorem [2] and Lemma [T]) seem quite natural, they are frequently mentioned 
in various publications, but the author did not see their complete proof. Therefore, the author 
propose her own proof of these “folk” propositions. 


3 Linearwise process 


3.1 Definition of (one-dimensional) linearwise Markov process 


Linearwise (or lineartype) processes are very useful in many problems of queueing theory. The 
fact is that the description of queueing system behaviour must contain a discrete component: for 
example, this discrete component can be a number of customers in the queueing system. And time 
intervals between changes of these discrete components are random variables. If these random 
variables have an exponential distribution, then the behaviour of queueing system is described by 
continuous-time Markov chain. 

But we refuse the condition of exponential distribution of time between state changes of the 
embedded Markov chain. Thus, the time between changes of the discrete components of gueueing 
process can be distributed arbitrarily. 

Let us formalize the above. 

Let ^ = lipjjll, i, j G Tl C Z be the transition matrix, and let <hfc(s) = P{Cfc ^ s} be the 
distribution functions such that $*,(0+) = 0, A: G 91. 


Remark 3. If $*,(0+) > 0, then we can modify the matrix ^ and the set 91 in such a way that 


replacement of the function $^( 0 ;) by the function 


$ fc(x) - d)fc(0+ ) 

1 - $fc(0+) 


does not change the behaviour 


of the corresponding stochastic process (see below). 

(m) ^ 

Let for all m G N and for all A: G 91, Q ' = Cfc be identically distributed random variables, and 
for a ^ 0 let (k,a be a random variable with the distribution function 


P{a,a ^ S} = <l>fc,,(s) = 1 - = P{a ^ 5 + a\Ck > a}; 

^k{a) 

It is evident that (k,o = Cfc (the designation “=” means the same distribution). 

If $fc(a) = 1, then we assume ^ q_ 

Let all random variables Cfc,a be mutually independent {k ^ m E Z,+ , a G IR+); 

superscript of random variable indicates that this is m-th exemplar of random variable (k] as 
already mentioned, = (k- 

Denote 3T = {{n,x)} = 91 X IK-j- 5 let the set be equipped with standard Borel cr-algebra 
RS{,2i'). The set ,2^ = {(n, x)} is the state space of linearwise Markov process. Let Si = {(i, •)} G 
be called i-th. level. 

Consider a Markov chain starting from the state r (i.e. = r). Let transition 

matrix of be Again we suppose that and random variables Ck^\ Ck,a are mutually 
independent. 
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Let us fix an arbitrary pair (no,xo) G ^ and denote 


to = 0, ti = Cno,a;o, 4 = Cno,xo + E C_^\no) , = max{i : U ^ t}, 


i=l 


nt 




(2) 


Definition 6. Linearwise Markov process with initial state Xq = {no,xo) is a stochastic process 
(Xt = {nt,Xt), t ^ O) defined by formula ([2]); its values are in the state space 

We skip the proof that the defined above process [Xt, t ^ O) is a Markov. 

Denote n{Xt) = nt, x{Xt) = Xt- Here nt is called the level where the process (X^, t ^ O) 

is located at the time t, vt is a moment of last change of process {Xt, t ^ 0) level: vt = sup{s < 
t : ng nt}. And Xt = (t — sup{s < t : n* 7^ nt}) is elapsed time of continuous being of process 
[Xt, t ^ 0) in the level nt before the time t; the times 4 are the times of pair {nt, Xt) = Xt change, 
or jumps of the process [Xt, t ^ O); note that Xt^,^ = 0. 

Construction (j2]) implies that at initial time t = 0, Xq = {no,xo), i.e. the process [Xt, t ^ O) 
starts from the state (no,xo). 

After the time t = 0, through the time Cno,xo, the process (X^, t ^ O) gets into state {i, 0) G St 
with probability Pno,i- Then the process (X*, f ^ O) stays in St in the time then, at the time 
{Cno,xo + C'^^) the process (X^, f ^ O) gets into state {j, 0) G Sj with probability pij, when it stays 
in the time Q , etc. 

We emphasize that random variable Cno,xo h a residual time of the process Xt stay in the level 
Sno, under the condition that until time t = 0 the process X* located continuously in the set Sn^ 
during the time Xq. We can interpret this situation as the start of process (X*, f ^ 0), 0 < 0, 
monitoring at the time f = 0, and the process (X^, f ^ O) is a sequel of process (X^, f ^ 0), 
0 < 0, started in some previous time 0. 

As already mentioned, linearwise processes (X^, t ^ O) often occur in the study of queueing 
systems. 

Remark 4. Usually for embedded Markov chain of linearwise process (X^, t ^ O) we have 
Pij = 0 if |i — j| 7^ 1. This fact makes the study of the process (X*, t ^ O) behaviour easier. 

3.2 Ergodic Theorem for linearwise Markov process 

3.2.1 Conditions 

1. Condition for Markov chain A4n- 

( E i{a^O)=A ] 

Vi,j em P < lim ^^- =pA=l- J2Pi = 

I J iGm 

let 01, = {i G 01: pi> 0} be a set of essential states of the Markov chain Mn- 

2 . Conditions for <hj(s): 

a. E 4 = Tj < cx) for all z G 01; 

b. E PiTi = E PiTi = T < 00 ; 

ie<n ieOTs 
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c. supp (Xt) = IJ supp (Ci)) ^ {a + bZ} = {a + bn,n E Z} for all a, & G M, where 

ieOTs 

supp (=5f(Cj)) is a support of random variable Q distribution. 


Remark 5. 

1. Condition 2.c. implies ^ D 0 > 0. This ensures that the process (Xf, t ^ O) is stochastic. 

i€<n 

We do not assume that < cx). 

2. Usually, for arbitrary Markov chain Xin the verihcation of condition 1. is difficult or even 
impossible. 

However, as a rule, the condition 1. can be verihed for queueing system. 

Usually, we can provide upper bounds for the values pi for queueing system linearwise process; 
as a rule, this estimation is enough to estimate the stationary distribution. 

3. Usually, for many queueing systems checking the condition 2.b. is not difficult. 


3.2.2 Theorem [2] 


Theorem 2. 

1 . Conditions 1-2 are necessary and sufficient for the existence of stationary distribution of the 
process (X*, t ^ O). 

2. Conditions 1-2 are necessary and sufficient for weak convergence of process (X*, t ^ O) distri¬ 
bution to this stationary distribution. 

3. Also, for stationary distribution of the process (X^, t ^ O) we have: 


= i, X > a ^ X* > b] = lim P{nt = i, Xt > a & x* > 6} 

t—>-oo 


oo 

^ j du, 

a+6 


(3) 


where x^ = (inf{s > t : Us ^ Ut} — t) is amount of overjump, or residual time of process (X^, t ^ 
O) continuous being in the level 3^ after the time t, i E 

Remark 6. The distribution given in ([3]) is stationary for the case, where supp (X*) C {a + bZj 
(naturally, with slight modihcations), but now we do not have sufficiently short and easy to read 
proof of this fact. 

3.2.3 Proof of Theorem [2] 

Proof. 

1. The necessity of conditions 1. and 2. is evident. 

2. Condition 1. implies existing of single indecomposable class of Markov chain This 

indecomposable class is a set of essential states of Markov chain This indecomposable 

class DTs C Tt is ergodic or periodic. 

Naturally, in the sequel we will consider essential states of Markov chain Xi^ only. 

3. Conditions 1., 2.a. and 2.b. imply 

lim P{nt = k} = (4) 
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^k,0 tfc,l 


>(^m) 

S/c 


tk,m 


tk:. 


Figure 1: Times of stay of the process (X^, t ^ O) in the level Sk and out of the level Sk- 


4. Conditions 1. and 2.c. imply: there exist gi,g2 £ such that random variable Cgi + Cq 2 is 
non-lattice. 

5. Let us hx an arbitrary fc G 01<j and (at the beginning of proof) let Xq = {k, 0). Denote 


4,0 = 0 , 


4 0 =^nf{t ^ 4,0 : nt^ k}; 

tk,j = inf{t ^ : nt = k}, - = inf{t ^ : rit 7^ /c}; 

aU) dsl + _ -j-t aL' _ p _ 4. 

4 — Sfc — ^k,j ^k,j- 

Here is the time when process (X^, t ^ O) hits into the level Sk, and t'j^^ is the time when the 
process (X^, t ^ O) leaves the level Sk] and is the length of time interval where X^ ^ Sk- in 
other words, is the time between two subsequent stays of the process (X^, t ^ O) in the set Sk- 


def . 


Evidently, random variables 6^^*^ are i.i.d. Denote by \l/fc(s) = P ^ s [> the distribution 


def 




function of 

Then 9^^'^ = 9^^'^ + is the time between two consequent (adjacent) hits of process (Xt, t ^ O) 
into the state (fc,0). The random variable 9^^'^ has the distribution function Tfc(s) = * ‘Ffc(s). 


Note that the random variable 9^^^ 


^ is non-lattice, since 

The paths of the Markov chain Mn from state {k} 
to the same state {k} contains the states {gi} and {^2} 


> 0 , 


- see condition 4. 

We emphasize that the process (X^, t ^ O) is a regenerative process, and times k eN, are 
regeneration points of the process (X*, t ^ O). 

6. Let us hx arbitrary time t and let us hnd P{nt = k,Xt> a,xl > b}- Denote 


Ek(t, m, u, dn) 


def 


Last hit of the process (X^, t ^ O) to the level Sk before 
the time t is the time tk^m, o>nd tk^m E {u, u + du) 


5 


Ek{t,m) = {Before the time t there was exactly m hits of the process (X^, t ^ O) into the level Sk} 

Ek{t, 0) = {Xt E Sk for all t E (0, t)}. 

Easy to see that 

{Ek{t, m, u, du) & Xt > a & > 6} -|- du ^t — a h > t — u + b^ , 

and for u ^ t — a we obtain 

Pjrit = k,Xt > a k. x} > b k. Ek(t, 0)} = <Ffc(t); 

P{nt = k,Xt > a,x} > b k Ek(t, m, u, du)} = $fc(t — u + b) dT™*(M); 

t—a 

P{nt = k, Xt > a, x} > b k Ek{t, m)} = J $fc(t — u + b) dT^*(n). 

0 
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Now denote Hk{u) = ^ and r = t — a. Thus, we get: 

m=l 

P{ni = k,Xt > a, xl > b} = P{nt = k,Xt > a,x’l > h Ek{t, 0)}+ 

t—a 


'Tiu) = 


^ P{nt = k,xt > a,x^ > b k Ek{t, m)} = - n + 6) dT^ 

m=l m=l Q 

r 

= $fc(t) + / <hfc(r + a + & - m) diffc(M). (5) 


Finally, let us find the limit in the formula ([5]) as t —?■ oo. Using Smith’s Theorem[T]for function 


5(r) = <Ffc(r + a + b), we get: 


/ <hfc(r + a + 6) dr / <hfe(r)dr 

lim P{nt = k,Xt > a, > 6} = 


t^OO 


f rdHk{r) 


E9, 


( 6 ) 


Then, combining (jl]) and (jb]), we obtain: 


f <hfc(r) dr 

rjn J '^ \ / rjn rjn 

lim Plui = k} = = ii]2i Pint = k,Xt > 0, xf > 0} = ^-= - K- P6k = —, 

+ ^ T t^ + oo ^0 ^0 Pfc 


and 

(r) dr. 

a-\-b 

7 . Now let Xq be arbitrary. Denote 0fc(Xo) = inf{t > 0 : Xt = (/c,0)}; it is easy to see that 
Qk{Xo) is a time of the first hit of the process (X*, t ^ O) into state (fc, 0) G Sk] this is the first 
regeneration time. 

Conditions 1. and 2 . a. imply P{0fc(Xo) < oo} = 1, since E0fc(Xo) < oo. Therefore for all 
initial states Xq we have: 


lim P{nt = k, Xt > a, xl > b} = ^ 

t—)- + oo i 


$ 


lim Pxoint = k,Xt> a,x} > b} = 

t—t+oo 


lim Pxo{nt = k,xt> a, xl > b k 0fc(Xo) G (n - 1, n]} = 

t—>-+oo 

n=l 

^ lim Pxo{nt = k,xt> a,x^ > b\Qk{Xo) e {n-l,n]}\Pxo{Qk{Xo) ^ {n-l,n\} 

n=l ^ ' 


Pk J <hfc(r)dr 


a-\-b 


T 


J2Pxo{Qk{Xo)e{n-l,n]} = 


Pk J ^kir)dr 


(i-\-b 


n=l 


T 


The Theorem [2] is proved. 


□ 









Remark 7. Theorem |2] is a generalization of Theorem [3l §2.6], where there are three variants of 
sujficient conditions for the proof of formula (l3l) . 

Remark 8. Using computations on the proof of Theorem [2] we can prove a fact very useful for 
applications of the renewal theory. This fact, in some sense, is “folklore”: many queueing theory 
experts know this fact, but we could not hnd when and who proved it. 


4 Expectation of amount of overjump 

Now let (Xt, f ^ O) be renewal process with non-lattice renewal period = Ci l^t the distribution 

k 

function of be d)(s); let be a renewal points. 

i=0 

Now we suppose E < cxo. 

Renewal process is a special case of linearwise Markov process, where embedded Markov chain 
has the single state only; stochastic matrix of this Markov chain is ^ = (1). We replace the 
state space of the process ^ = {1} x M+ by ^ = M+ (we reduce the full states space of the 
process t ^ O) by dropping not changing hrst component). The amount of overjump x* 
of renewal process at the time t is the time elapsing from the time t to the next renewal (fj): 
X* = (min{tj : ti > t} — t). 

It is well known (n. 0). that if the distribution of C, is non-lattice and E(C < cxd, then 

oo_ 

f ^(u) du 

lim P{x* > s} = P{x* > s} = ^—pt—; random variable xj' is called stationary amount of 

t—>-oo ^ 

EC' 

overjump, and Ex^ = . 

2E ^ 

Lemma 1. If the distribution of ( is non-lattice, and EC' < oo, then Ex* ^ Ex* = 

Proof. Usually ([H §2.3], [21 §2.6]) the proof of formula ([I]) follows from the equality 


R(s, t) = P{x* > s} = ^{t + s)+ I ^{t-u + s) dH{u). 


(7) 


This equality ([7]) is a special case of (|5]), where H{u) = ^ <h**(n) is (nondecreasing) renewal 

i=l 

function. For A > 0 we have: 

R{s, t A) — R{s, t) = <F(t + s) — d)(t -|- A -|- s)-|- 

t t+A 

^{t -\- A — u -I s) — <F(t — u-\- s) dH(u) -\- / <F(f -|- A — u -|- s) dH{u) ^ 0, 


i.e. 


R{s, t -\- A) = P{x 4 +^ > s} ^ R{s, t) = Plxj > s}. 


(8) 


therefore for ti < ^2 

OO OO OO 

Ex*^ = J R{s,ti)ds^ j i?(s, ^2) ds = E x*2 ^ hm E x^ = J 


^ f 1 — <F(m) du^ 


/ 1 — <F(m) du 
0 


ds = 


EC' 

2^’ 
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The Lemma [His proved. 

Remark 9. In addition, from ([8]) we have: 


□ 


1. If < oo for k > 2, then 


( 




{k — 1) J t) ds ^ {k — 1) J ^ 

0 0 


/ 1 — $(m) dti 




EC 


V 


ds = 


/ 


oo oo 


f ^ d /(I — $(«)) dti 

_o_ s _ 

EC 


Ec^ 

A:EC‘ 


2. If Ee"^ < oo for a > 0, then 


OO OO 


a 


E = a J e°'^R{s, t) ds ^ — J e"^ J ~ ^ 

0 Os 


EC 


-EC+ / e“"(l-$(s))ds = 


Ee"^ 


- 1 . 


Remark 10. Recall that Xt is the time elapsed from the previous renewal to t, i.e. Xt = (f—max{fj : 
ti < t}). 

Random variable Xt is called stationary amount of underjump, if lim ^{xt > s} = P{Xi > s}. 


t^OO 


J ^(u) du 

From Theorem 2 we have PjTt > s} = ° —. 

The amount of underjump xt is a single component of the process (^Xt, t ^ O). And for 
a similar proposition is true: if the distribution of C is non-lattice, and EC^ < oo, then Ext 

The proof is identical to the proof of Lemma [T] 

Corollary 2. If we assume that E (C*)^ < oo in the conditions of Theorem [H, then for all 
r > inf{f ^ 0 : Xt = (A:,0)} the inequalities 


E {Xr\nr = k) ^ 


E(C;. 


2ECfe ’ 


E {xl\nr = k) ^ 


E(Cfc 


2ECa: 


(9) 


hold. 

These inequalities give the possibility to obtain not only estimation of linearwise process sta¬ 
tionary distribution, but also an estimation of linearwise process distribution at the arbitrary (big 
enough) time. Naturally, we can not use ^ before the first regeneration time. 
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5 Multidimensional piecewise-linear Markov processes 


Now the great interest of queueing theory is the study of multichannel queueing system and of 
queuing network. Let some queueing system (or queuing network) consist of iV ^ oo servers 
and incoming flow of a customers can be also multichannel, i.e. incoming flow includes iL > 0 
(obviously independent) incoming flow of customers; obviously all incoming flows are independent. 

Assume that the time interval between customer arrivals of Ath incoming flow has distribution 
function $j(s), in the Ath server the time of the service has distribution function Fi{s). We can 
also consider some additional conditions, such as: 

- customers can have different types of service; 

- some customers may be “impatient” (i.e., this customer can leave the queueing system before 
completion of service if (possibly random) residence time in the system has ended); 

- and so on. 

The behaviour of multichannel and multiserver system can be described by Markov process 

( OO OO \ / OO 

IJ IJ {N” X M™} j = <^o[j [ U '^n,m 

n=l m=K+l J \n=l 

flere we show a set which is knowingly sufficient to describe the behaviour of the complex 
queueing system: in fact the state space ^ may be easier. 

For example, the state of process [Xf, f ^ O) at the time t can be described by vector 


X, = 


Vi 


(K) Jl) 


X 


,x 


(Mt) .,( 1 ) 




m (1) 

, / Lj- 


,n 


(Qt) 


Here A/) is a number 


of customers in queueing system, Xit is a number of customers served at time t; is the time 
from the last arrival of the customer from j-th incoming flow; x[^'^ is elapsed time of service on 
j-th server, i.e. elapsed time of service of customer located on j-th server; also we can include into 
vector Xf service waiting time of customers or duration of stay in the system for all customers; 
total length of the queue of queueing system or queue length on each server of queueing system; 
possibility of packaged service of incoming customers, and so on.: some of this parameters are pre¬ 
sented in parts of the vector Xt - that is ..., ) (elapsed server time) and (n[^\ ..., ) 


(queue length). 

Continuous components of process (W, t ^ O) change linearly between the times of process 
(Xj, f ^ O) discrete components change: at the time of discrete components change some of 
continuous components leave the vector Xt, and some of new continuous components may appear 
in the vector Xt. For each specific queuing systems we can define the transition probabilities for 
the vector Xt'. process (X^, t ^ O) is Markov for a suitable choice of the vector Xt component; we 
shall say that this process (X^, t ^ O) is called piecewise-linear Markov process. 

The “level” of such piecewise-linear Markov process (as described above) can be dehned as A/), 
or the pair {Aft, Ait), or more comprehensive set of discrete components of the vector Xt. 

Then, in general case, the embedded chain of “levels”, in which process can be staying, is 
not homogeneous Markov chain. However, in some cases it is possible to define upper hound of 
this embedded non-homogeneous chain transition probabilities by transition probabilities of some 
homogeneous Markov chain. 

Indeed, to hnd such a bound, for example, we can consider another auxiliary queueing system 
having embedded homogeneous Markov chain; but the parameters of this new queueing system 
are the bounds for the parameters of original queueing system. 

Further, after studying this new auxiliary queueing system, we can prove the ergodic theorem for 
auxiliary queueing system. Then, we can obtain formulas of auxiliary queueing system stationary 
distribution. Using these formulas, we can find bounds for stationary distribution of original 
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queueing system; in fact, we prove the ergodic theorem for original queueing system. 

Moreover, using these bounds we can hnd an estimate for convergence rate of original queueing 
system distribution to stationary distribution. 

Also hnding of such bounds for stationary distribution of multidimensional piecewise-linear 
Markov processes is usually possible for the following reason. Usually, incoming flow of queueing 
system is such that only transitions A/i A/i + 1, A4t A4t + 1, Aft JVt — A4t A4t — f 
{i ^ 1) are possible (only these transitions have positive probability). 

So, let for some Af the inequality p„ = P{ni— )-n + l} ^p< | is satished for all n ^ Af. And 
let the set of distributions $i,..., ^k, Fi, - ■ ■, Fj^ has common non-lattice support. Then we can 
establish: for any initial state (or initial distribution) of the process (Xt, t ^ O), the distribution 
of this process (X^, t ^ O) converges weakly to stationary distribution 

Now, we can use the technique machinery applied for the proof of Theorem [21 and also we 
can apply well-known facts from the random walk theory. This way, we can obtain estimates for 
stationary distribution, for example, such that: 


m oo 

n/(i-nw)di, 

^(A{k)) $ (27r)‘“^ for k > M', m,ai ,... ,o„)) ^ -for k^M, 

0 i=l 

where A{k) = {X G ^ : A/" = k}, and B{k,m,ai,..., am) = {X G ^ : Af = k,AA = m,xi > 

01, . . . , Xm ^ }, 

Such inequalities can be applied for estimating the convergence rate of process (X^, t ^ O) 
distribution to stationary distribution ^ of the process (X^, t ^ O'), and for estimating some 
parameters of queueing system behaviour in the long range of time. 
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